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DETAILED ACTION 

This action is in response to the communication filed on February 7, 2007. 

• Claims 1-3, 7, 11, 12, and 25-31 have been amended. 

• Claim 22 has been cancelled. 

• Claim 32 has been added. 

Claims 1-21 and 23-32 are pending and have been examined. Previous objections and 
rejections not included in this office action have been withdrawn. 

Specification 

1 . The disclosure is objected to because it contains an embedded hyperlink (on 
page 13) and/or other form of browser-executable code. Applicant is required to delete 
the embedded hyperlink and/or other form of browser-executable code. See MPEP § 
608.01. 

Claim Rejections - 35 USC § 102 

2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

3. Claims 1-21 and 23-32 are rejected under 35 U.S.C. 102(b) as being anticipated 
by Zimmerman (U.S. Patent Application Publication 2002/0193975). 
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Claim 1 

As per claim 1, Zimmerman teaches a method for assisting navigation of 
digital content using a tangible medium, comprising: receiving an instruction to 
access digital content corresponding to a portion of a tangible medium (in figure 1 
and in paragraph 20 on page 2, "The system generally comprises a probe 
device and a computer. The probe is manipulated over off-line 
media, preferably printed medium, and information is transmitted 
to the computer and matched to electronic media"), said medium being 
readable by a user-positionable input device (in figure 1 and in paragraph 20 on 
page 2, "The system generally comprises a probe device and a 
computer. The probe is manipulated over off-line media, 
preferably printed medium, and information is transmitted to the 
computer and matched to electronic media"), and said digital content 
being accessible from a stored file (in paragraph 1 18 on page 7, " All 
programming and data related thereto are stored in computer 
memory, static or dynamic, and may be retrieved by the user in 
any of: conventional computer storage (local or remote), display 
(i.e., CRT) and/or hard copy (i.e., printed) formats"); determining 
and accessing stored digital content corresponding to said input device's 
instantaneous position on said tangible medium (in paragraph 21 on page 2, 
"Based upon the position of the probe oyer the off-line media, 
corresponding multimedia data is retrieved from storage and 
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presented to the user"); and enabling electronic navigation of said digital 
content (in paragraph 21 on page 2, "Preferably, the multimedia data and 
off-line media are designed so as to provide an educational 
experience in which the synchronization of the probe position 
and output provides the visceral experience normally experienced 
by a user of a real tool. To simulate the real tool, the 
multimedia data corresponds to the output a real tool would 
produce when the real tool is used to inspect the items on the 
off-line media "), including enabling toggling between browsing of said 
tangible medium and browsing of said digital content on a computer screen using 
said input device (in paragraphs 24, 77, 81, and 89-90). 

Claim 2 

As per claim 2, the rejection of claim 1 is incorporated and Zimmerman further 
teaches determining a change in position of said input device on said tangible 
medium; and obtaining a new stored file corresponding to said change in 
position (in paragraph 24 on page 2, "the printed medium additionally has 
a number of icons representing different tools to be simulated. 
By placing the probe over a particular icon , the tool 
represented by the icon is selected such that the retrieved 
multimedia data corresponds to the selected tool "). 
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Claim 3 

As per claim 3, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said determining and accessing stored digital content includes: 
obtaining digital signals representing a localized region of said tangible medium, 
said localized region being proximate to said position of said input device on said 
tangible medium (in paragraph 66 on page 4, "Referring to fig. 4, in a 
preferred embodiment of the present invention, probe 104 
contains a CMOS monochromatic camera 4 00 with wide angle lens 
406 and illumination system 410 to capture an image segment 408 
printed on the medium 106, underneath the probe . The image is 
detected and transmitted to the computer 100 by image processor 
4 02 , communicating to the computer 100 through connector 404, 
such as Universal Serial Bus (USB) "; the computer inherently obtains digital 
signals through a USB connector); determining at least one stored file 
corresponding to said localized region, and containing said digital content, by 
using pattern matching (in paragraph 66 on page 4, "The image segment 408 is 
a portion of a full image 412 appearing in the off-line media 
106. An image retrieval method running in the computer 100 
receives the image segment 408, and outputs image identification 
and position information of the image segment 4 08" and in paragraph 68 
on page 4, "FIG. 5 shows a flow chart summarizing the methods of 
sampled image to reference image matching... At block 506 the 
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salient features of the sampled image are extracted. At block 
508 the salient features of the sampled image are used to find 
the closest reference image " in view of figure 5; comparing salient features is 
one form of pattern matching); and retrieving an appropriate portion of said file to 
enable user navigation (in paragraph 21 on page 2, "Based upon the position 
of the probe over the off-line media, corresponding multimedia 
data is retrieved from storage and presented to the user... To 
simulate the real tool, the multimedia data corresponds to the 
output a real tool would produce when the real tool is used to 
inspect the items on the off-line media "). 

Claim 4 

As per claim 4, the rejection of claim 3 is incorporated and Zimmerman further 
teaches wherein said pattern matching is based on correlating a pattern within 
said localized region with a pattern in said stored file (in paragraph 68 on page 4, 

"FIG. 5 shows a flow chart summarizing the methods of sampled 
image to reference image matching. . . At block 504 the sampled 
image is captured by the camera 4 00 in the probe 104. At block 
506 the salient features of the sampled image are extracted. At 
block 508 the salient features of the sampled image are used to 
find the closest reference image " in view of figure 5). 
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Claim 5 

As per claim 5, the rejection of claim 3 is incorporated and Zimmerman further 
teaches wherein said pattern matching is based on correlating a pattern 
embedded within said medium itself (in paragraph 20 on page 2, "The system 
generally comprises a probe device and a computer. The probe is 
manipulated over off-line media, preferably printed medium , and 
information is transmitted to the computer and matched to 
electronic media" and in paragraph 45 on page 3, "Printed, off-line 
medium 106 has a suitable image 108 imprinted thereon"). 

Claim 6 

As per claim 6, the rejection of claim 3 is incorporated and Zimmerman further 
teaches wherein said tangible medium was previously created independently of 
said file (in paragraph 87 on page 6, "in an application of the invention . 
where the media 106 includes text, the user may circle text, or 
swipe the text with the probe 104 to select, translate, provide 
definition (e.g. from a dictionary), underline, highlight, make 
bold, copy or cut the corresponding electronic text appearing in 
the computer 100. In this application, images of the text, or 
salient features of the text need not be stored in the computer 
100. In a preferred embodiment, text is stored as ASCII 
characters along with the font and other layout and contextual 
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information to enable the computer to generate an accurate image 
of the page, or otherwise sufficiently represent the page"). 

Claim 7 

As per claim 7, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said determining and accessing stored digital content includes: 
obtaining coordinates of said position of said input device on said tangible 
medium, determining at least one stored file corresponding to said position and 
containing said digital content, determining coordinates within said stored file, 
corresponding to said input device position coordinates, by using coordinate 
mapping ("The computer decodes the bar code information into probe 
position and image identification , resulting in a synchronized 
image 204 on the computer screen of human bones 206" in view of figures 
2a and 2b; identifying a position on an image inherently involves obtaining and 
determining coordinates, and in this example, the probe position (coordinate) on the 
tangible medium is directly mapped to a position on the stored image); and using said 
determined coordinates to retrieve an appropriate portion of said file to enable 
user navigation ("... resulting in a synchronized image 204 on the 
computer screen of human bones 206" in view of figure 2b). 
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Claim 8 

As per claim 8, the rejection of claim 7 is incorporated and Zimmerman further 
teaches wherein said coordinate mapping involves a linear transformation from 
tangible medium coordinates to stored file coordinates (in paragraph 51 on page 3, 

"The computer decodes the bar code information into probe 
position and image identification, resulting in a synchronized 
image 2 04 on the computer screen of human bones 206" in view of figures 
2a and 2b; in this example, the probe position on the tangible medium is directly 
mapped to the position on a stored image, and therefore inherently involves a linear 
transformation from tangible medium coordinates to stored file coordinates). 

Claim 9 

As per claim 9, the rejection of claim 7 is incorporated and Zimmerman further 
teaches wherein at least one of said tangible medium and said stored file includes a grid 
system (in paragraph 51 on page 3, "The computer decodes the bar code 
information into probe position and image identification, 
resulting in a synchronized image 204 on the computer screen of 
human bones 206" in view of figures 2a and 2b; identifying a position on an image 
inherently involves a coordinate or grid system). 
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Claim 10 

As per claim 10, the rejection of claim 7 is incorporated and Zimmerman further 
teaches wherein said determining said stored file includes utilizing a file index 
read from said tangible medium (in paragraph 49 on page 3, "fig. 2a 
illustrates the use of the present invention with position and 
image identification information encoded in bar codes printed on 
the off -line media "). 

Claim 1 1 

As per claim 1 1 , the rejection of claim 7 is incorporated and Zimmerman further 
teaches wherein a file index was previously generated during creation of said 
tangible medium (in paragraph 49 on page 3, "FIG. 2a illustrates the use 
of the present invention with position and image identification 
information encoded in bar codes printed on the off-line media "). 

Claim 12 

As per claim 12, the rejection of claim 7 is incorporated and Zimmerman further 
teaches wherein said tangible medium includes a plurality of machine-readable 
patterns embedded in said medium itself (in paragraph 45 on page 3, "Printed, 
off-line medium 106 has a suitable image 108 imprinted thereon" 
and in paragraph 49 on page 3, "fig. 2a illustrates the use of the 
present invention with position and image identification 
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information encoded in bar codes printed on the off-line media "); 
said obtaining coordinates of said position of said input device is based on 
reading a unique pattern at said position of said input device, and analyzing said 
unique pattern to determine said coordinates (in paragraph 51 on page 3, "The 
probe 104 is placed over the picture of the hand 200 in book. 
The probe 104 sends bar code information to the computer. The 
computer decodes the bar code information into probe position 
and image identification , resulting in a synchronized image 204 
on the computer screen of human bones 206" in view of figures 2a and 2b). 

Claim 13 

As per claim 13, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said digital content includes an image, and said navigation 
includes displaying said image (in paragraph 81 on page 5, "The invention 
matches the sampled image 706 to a reference image of the frog's 
eye retrieved from storage device 102. Based on the probe 104 
position, reference image, and tool selected, a magnified view 
of a frog eye 802 is presented on the computer screen 800. The 
invention synchronizes the movement of the probe 104 to the 
display of the frog's eye 802. When the user moves the tool to 
the right, the image of the frog's eye on the computer 800 
screen would also move to the right, giving the user the 
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visceral experience of holding an actual microscope over a 
frog 1 s eye" in view of figure 7). 

Claim 14 

As per claim 14, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said digital content includes audio, and said navigation includes 
playing said audio (in paragraph 77 on page 5, "when the user 616 touches a 
particular word,, for example "forest 11 618 as show in FIG. 6, the 
computer 610 retrieving an audio clip of the spoken word 
"forest" from storage 614, and played it out speakers 620. When 
the user 616 touches the drawing 606, the sound of a bear 
growling occurs" in view of figure 6). 

Claim 15 

As per claim 14, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said tangible medium serves as a video storyboard (in paragraph 

89 on page 6, "The remote server finds the best reference image 
match. In response to this match, the remote server sends a 
video clip 914 (for example MPEG1 compressed video) to the 
computer for presented on the computer display 912, in response 
to the sampled image in the magazine 904" in view of figure 9a and 9b). 
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Claim 16 

As per claim 16, the rejection of claim 1 is incorporated and Zimmerman further 
wherein said navigation includes at least one user-selectable mode (in paragraph 

83 on page 5, "Since the simulated display on the computer screen is 
not bound by the static reality of off-line media, the 
synchronized presentation may include dynamic images. For 
example when viewing the eye, the user can press a button on the 
probe 104 (not shown) to generate a virtual bright light at the 
probe, causing the image of the frog's eye 802 to retract in the 
socket"). 

Claim 17 

As per claim 17, the rejection of claim 16 is incorporated and Zimmerman further 
teaches wherein said modes are designated on, and selectable from, said tangible 
medium (in paragraph 24 on page 2, " the printed medium additionally has 
a number of icons representing different tools to be simulated. 
By placing the probe over a particular icon, the tool 
represented by the icon is selected such that the retrieved 
multimedia data corresponds to the selected tool " and in paragraph 66 
on page 4, "A presentation method running in the computer 100 
receives the image segment 4 08 information and presents 
multimedia content contained in a CD-ROM storage device 102 
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based on the virtual tool or function selected by the user " in view 
of figure 4). 

Claim 18 

As per claim 18, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said tangible medium includes paper (in paragraph 21 on page 2, 

"The probe is manipulated over off-line media, preferably printed 
medium , " and in paragraph 26 on page 2, "Off-line media can be any non- 
networked media or object including books , magazines , 
newspapers, posters, pictures , mosaics, tapestry, two and three 
dimensional objects, animals, people, furniture, toys, cups, 
plates, silverware, business cards , and clothing"). 

Claim 19 

As per claim 19, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said input device includes an optical device (in paragraph 2 on 

page 2, "The probe includes a camera to capture image segments and 
transmits them to a computing device"). 

Claim 20 

As per claim 20, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said input device includes a radio frequency device (in paragraph 



Application/Control Number: 10/752,786 Page 15 

Art Unit: 2178 

23 on page 2, " The probe includes a camera to capture image segments 
and transmits them to a computing device , either local or 
remote, communicating through wired or wireless means " and in 
paragraph 45 on page 3, "... Probe device 104 is attached to computer 
100 via any suitable I/O interface so as to allow the probe to 
input data to computer 100. The I/O interface can including 
wired (e.g. USB, serial, parallel, firewire, optical 
communication) or wireless (e.g. 802.11, Bluetooth, UHF, 
infrared, CDMA, G3, PCS, mobile phone, ISM band RF) means "). 

Claim 21 

As per claim 21 , the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said tangible medium is two-dimensional, yet includes three- 
dimensional information (in paragraph 26 on page 2, "Of f- line media can be 
any non-networked media or object including books, magazines, 
newspapers, posters, pictures ,, mosaics, tapestry, two and three 
dimensional objects, animals, people, furniture, toys, cups, 
plates, silverware, business cards, and clothing..."). • 

Claim 23 

As per claim 23, the rejection of claim 1 is incorporated and Zimmerman further 
teaches wherein said tangible medium was previously created using said stored 



Application/Control Number: 10/752,786 Page 16 

Art Unit: 2178 

file (in paragraph 50 on page 3, "the bar codes 202 are printed on clear 
plastic (e.g. mylar) sheets and adhered to the pages of the off- 
line media. This allows a book to be published using 
conventional printing techniques, and bar codes to be placed on 
the pages after conventional printing "; the bar codes refer to the stored 
files likely used in the printing of the book). 

Claim 24 

As per claim 24, the rejection of claim 1 is incorporated and Zimmerman further 
teaches implemented in a handheld portable electronic device ("the present 
invention may be implemented on a conventional IBM PC or 
equivalent, mobile phone , personal digital assistant (PDA) , 
tablet computer , multi-nodal system (e.g., LAN) or networking 
system (e.g., Internet, WWW)"). 

Claim 25, 26, and 27 

Claims 25, 26, and 27 are the computer-readable storage medium claims 
corresponding to the method claims 1, 3 and 7 respectively, and are rejected under the 
same reasons set forth in connection with the rejection of claims 1 , 3, and 7. 

Zimmerman further teaches the computer-readable storage medium 
comprising logic instructions (in paragraph 1 18 on page 7, "The above system 
and its described functional elements are implemented in various 
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computing environments... All programming and data related 
thereto are stored in computer memory , static or dynamic, and 
may be retrieved by the user. . ."). 

Claim 28 

Claim 28 is the system claim corresponding to the method claim 1 and is rejected 
under the same reasons set forth in connection with the rejection of claim 1. 

Zimmerman further teaches the means for receiving (in paragraph 45 on page 

3, "Probe device 104 is attached to computer 100 via any suitable 
I/O interface so as to allow the probe to input data to computer 
100. The I/O interface can including wired (e.g. USB, serial, 
parallel, firewire, optical communication) or wireless (e.g. 
802.11, Bluetooth, UHF, infrared, CDMA, G3 , PCS, mobile phone, 
ism band RF) means" and in paragraph 118 on page 7, "For example, the 
present invention may be implemented on. . . multi -nodal system 
(e.g., LAN) or networking system (e.g., Internet, WWW) "), means for 
determining and accessing (in paragraph 23 on page 2, "The computing device 
analyses the image segments, determines the image identity, and 
retrieves and presents electronic media corresponding to the 
image to the user"), and means for enabling (in paragraph 81 on page 5, "Based 
on the probe 104 position, reference image, and tool selected, a 
magnified view of a frog eye 8 02 is presented on the computer 
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screen 800. The invention synchronizes the movement of the probe 
104 to the display of the frog's eye 802")., Therefore, Zimmerman 
teaches the system claimed. 

Claim 29, 30, and 31 

Claims 29, 30, and 31 are the system claims corresponding to the method claims 
1 , 3 and 7 respectively, and are rejected under the same reasons set forth in connection 
with the rejection of claims 1 , 3, and 7. 

Zimmerman further teaches the interface (in paragraph 45 on page 3, "Probe 
device 104 is attached to computer 100 via any suitable I/O 
interface so as to allow the probe to input data to computer 
100. The I/O interface can including wired (e.g. USB, serial, 
parallel, firewire, optical communication) or wireless (e.g. 
802.11, Bluetooth, UHF, infrared, CDMA, G3 , PCS, mobile phone, 
ISM band RF) means") and processor (in paragraph 23 on page 2, "The 
computing device analyses the image segments, determines the 
image identity, and retrieves and presents electronic media 
corresponding to the image to the user"; a computing device inherently 
comprises a processor). Therefore, Zimmerman teaches the system claimed. 



Claim 32 (New) 



Application/Control Number: 10/752,786 Page 19 

Art Unit: 2178 

As per claim 32, Zimmerman teaches a method for assisting navigation of 
digital content using a tangible medium, comprising: receiving an instruction to 
access digital content corresponding to a portion of a tangible medium (in figure 1 
and in paragraph 20 on page 2, "The system generally comprises a probe 
device and a computer. The probe is manipulated over off-line 
media, preferably printed medium, and information is transmitted 
to the computer and matched to electronic media"), said medium being 
readable by a user-positionable input device (in figure 1 and in paragraph 20 on 
page 2, "The system generally comprises a probe device and a 
computer. The probe is manipulated over off-line media, 
preferably printed medium, and information is transmitted to the 
computer and matched to electronic media"), and said digital content 
being accessible from a stored file (in paragraph 118 on page 7, " All 
programming and data related thereto are stored in computer 
memory, static or dynamic, and may be retrieved by the user in 
any of: conventional computer storage (local or remote), display 
(i.e., CRT) and/or hard copy (i.e.> printed) formats"); determining 
and accessing stored digital content corresponding to said input device's 
instantaneous position on said tangible medium (in paragraph 21 on page 2, 
"Based upon the position of the probe over the off-line media, 
corresponding multimedia data is retrieved from storage and 
presented to the user"); enabling electronic navigation of said digital content 
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(in paragraph 21 on page 2, "Preferably, the multimedia data and off- 
line media are designed so ^s to provide an educational 
experience in which the synchronization of the probe position 
and output provides the visceral experience normally experienced 
by a user of a real tool. To simulate the real tool, the 
multimedia data corresponds to the output a real tool would 
produce when the real tool is used to inspect the items on the 
off-line media "), and enabling use of multiple tangible media to facilitate 
three-dimensional navigation (in paragraph 72 on page 5, "when three 
sequential sampled images are identified as belonging to the 
same reference image, a match is declared", in paragraph 22 on page 7, 
"the multimedia data and off-line media is. designed so as to 
provide a commerce experience in which the synchronization of 
the off-line media and electronic material enhances the 
knowledge of the user to the products in the off-line media, for 
example, showing different views or colors of a dress, or a 
simulation of the user in the dress ", in paragraphs 26, 63, 81 and 82, and 
figure 1). 

Response to Arguments 

4. Applicant's arguments filed February 7, 2007 have been fully considered but they 
are not persuasive. 
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Applicant argues, in substance, that 1) Zimmerman allegedly fails to disclose or 
suggest the step of enabling a user to toggle between browsing of the tangible medium 
and browsing of the digital content on the computer screen using the input device. 2) 
Zimmerman allegedly fails to disclose or suggest using multiple tangible media to 
facilitate three-dimensional navigation. However, examiner respectfully disagrees. 

Zimmerman teaches enabling toggling between browsing of the tangible medium 
and browsing of the digital content on the computer screen using the input device in 
paragraphs 24, 77, 81, and 89-90, for example. In these paragraphs, the user can 
move a finger, a probe or other device over the tangible medium onto a particular 
section, icon, word, etc. of the tangible medium, browsing the tangible medium. 
Through this, digital content is retrieved which toggles to the browsing of the digital 
content (e.g. moving the frog's eye around on the computer screen, playing an audio or 
video clip). The user can then select a different section, icon, word, etc., toggling back 
to browsing of the tangible medium. 

Zimmerman also teaches using multiple tangible media to facilitate three- 
dimensional navigation. Images are a form of tangible media because they are capable 
of being perceived. Zimmerman teaches using three sampled images to retrieve digital 
content to be navigated (in paragraph 72 on page 5). If applicant considers "tangible 
media" as only physical objects, Zimmerman also teaches it. For example, Zimmerman 
teaches using a digital pen, a printed media, a storage device and a display to facilitate 
navigation of retrieved digital content (in figure 1). Zimmerman also teaches three- 
dimensional navigation for the retrieved digital content (in paragraph 22 on page 7, 
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simulation of the user in the dress). Moreover, Zimmerman teaches navigating on a 
plane (in paragraphs 63 and 81) and zooming (in paragraphs 81 and 82), which covers 
three-dimensions, as well as navigating three-dimensional objects (in paragraph 26). 
Therefore, Zimmerman teaches enabling use of multiple tangible media to facilitate 
three-dimensional navigation. 

5. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 . 1 36(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to William Wong whose telephone number is 571-270- 
1399. The examiner can normally be reached on M-F 7:30-5:00 EST with every other 
Friday 7:30-4. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Stephen Hong can be reached on 571-272-4124. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 





STEPHEN HONG 
SUPERVISORY PATENT EXAMINER 



